Skip to content

[airflow] Update oudated AIR301, AIR302 rules#17123

Merged
ntBre merged 15 commits intoastral-sh:mainfrom
astronomer:update-AIR302-303-rules
Apr 7, 2025
Merged

[airflow] Update oudated AIR301, AIR302 rules#17123
ntBre merged 15 commits intoastral-sh:mainfrom
astronomer:update-AIR302-303-rules

Conversation

@Lee-W
Copy link
Contributor

@Lee-W Lee-W commented Apr 1, 2025 •
edited
Loading

Summary

Some of the migration rules has been changed during Airflow 3 development. The following are new AIR302 rules. Corresponding AIR301 has also been removed.

  • airflow.sensors.external_task_sensor.ExternalTaskMarker → airflow.providers.standard.sensors.external_task.ExternalTaskMarker
  • airflow.sensors.external_task_sensor.ExternalTaskSensor → airflow.providers.standard.sensors.external_task.ExternalTaskSensor
  • airflow.sensors.external_task_sensor.ExternalTaskSensorLink → airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink
  • airflow.sensors.time_delta_sensor.TimeDeltaSensor → airflow.providers.standard.sensors.time_delta.TimeDeltaSensor
  • airflow.operators.dagrun_operator.TriggerDagRunLink → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
  • airflow.operators.dagrun_operator.TriggerDagRunOperator → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator
  • airflow.operators.python_operator.BranchPythonOperator → airflow.providers.standard.operators.python.BranchPythonOperator
  • airflow.operators.python_operator.PythonOperator → airflow.providers.standard.operators.python.PythonOperator
  • airflow.operators.python_operator.PythonVirtualenvOperator → airflow.providers.standard.operators.python.PythonVirtualenvOperator
  • airflow.operators.python_operator.ShortCircuitOperator → airflow.providers.standard.operators.python.ShortCircuitOperator
  • airflow.operators.latest_only_operator.LatestOnlyOperator → airflow.providers.standard.operators.latest_only.LatestOnlyOperator
  • airflow.sensors.date_time_sensor.DateTimeSensor → airflow.providers.standard.sensors.DateTimeSensor
  • airflow.operators.email_operator.EmailOperator → airflow.providers.smtp.operators.smtp.EmailOperator
  • airflow.operators.email.EmailOperator → airflow.providers.smtp.operators.smtp.EmailOperator
  • airflow.operators.bash.BashOperator → airflow.providers.standard.operators.bash.BashOperator
  • airflow.operators.EmptyOperator → airflow.providers.standard.operators.empty.EmptyOperator

closes: #17103

Test Plan

The test fixture has been updated and checked after each change and later reorganized in the latest commit

@Lee-W Lee-W marked this pull request as draft April 1, 2025 15:05
@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2025 •
edited
Loading

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+6 -2 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+6 -2 violations, +0 -0 fixes) 

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ airflow-core/tests/unit/utils/test_dot_renderer.py:103:22: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
+ airflow-core/tests/unit/utils/test_dot_renderer.py:83:22: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
- dev/airflow_perf/dags/perf_dag_1.py:41:10: AIR302 Import path `airflow.operators.bash_operator` is moved into `standard` provider in Airflow 3.0;
+ dev/airflow_perf/dags/perf_dag_1.py:41:10: AIR302 `airflow.operators.bash_operator.BashOperator` is moved into `standard` provider in Airflow 3.0;
- dev/airflow_perf/dags/perf_dag_1.py:48:12: AIR302 Import path `airflow.operators.bash_operator` is moved into `standard` provider in Airflow 3.0;
+ dev/airflow_perf/dags/perf_dag_1.py:48:12: AIR302 `airflow.operators.bash_operator.BashOperator` is moved into `standard` provider in Airflow 3.0;
+ performance/src/performance_dags/performance_dag/performance_dag.py:111:13: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
+ providers/ydb/tests/system/ydb/example_ydb.py:111:23: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
AIR302 8 6 2 0 0

@Lee-W Lee-W changed the title Update air302 303 rules [airflow] update oudated AIR302, AIR303 rules Apr 1, 2025
@Lee-W Lee-W mentioned this pull request Apr 1, 2025
Closed
2 tasks
@Lee-W Lee-W force-pushed the update-AIR302-303-rules branch from 2c0eaf2 to 117f1ae Compare April 1, 2025 15:28
@Lee-W Lee-W mentioned this pull request Apr 2, 2025
Closed
6 tasks
@Lee-W Lee-W force-pushed the update-AIR302-303-rules branch from 117f1ae to b7be1f4 Compare April 2, 2025 10:02
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 2, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
cc95258 
…#17123
@Lee-W Lee-W force-pushed the update-AIR302-303-rules branch 3 times, most recently from 914825c to 37057cd Compare April 3, 2025 02:58
@Lee-W Lee-W changed the title [airflow] update oudated AIR302, AIR303 rules [airflow] update oudated AIR301, AIR302 rules Apr 3, 2025
@Lee-W Lee-W marked this pull request as ready for review April 3, 2025 02:59
@ntBre ntBre self-assigned this Apr 3, 2025
@Lee-W
Copy link
Contributor Author

Lee-W commented Apr 4, 2025

@ntBre Would be nice if we can get a quick review on this one 🙂 Thanks 🙏

@ntBre
Copy link
Contributor

ntBre commented Apr 4, 2025

Will do! Feel free to ping me or request my review on any of these. There's a chance I could have missed them before taking over from Dhruv. I assigned myself to a few last night :)

ntBre
ntBre approved these changes Apr 4, 2025
View reviewed changes
Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a few nits/questions

Lee-W added 12 commits April 7, 2025 09:06
@Lee-W
feat(AIR302): move the following rules from AIR301 to AIR302 due to A…
21068d4 
…irflow 3 change

* ``airflow.operators.bash.BashOperator`` → ``airflow.providers.standard.operators.bash.BashOperator``
* ``airflow.operators.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
@Lee-W
test(AIR301): remove DummyOperator and EmptyOperator from AIR301 test…
c4cf780 
… cases
@Lee-W
test(AIR302): add test cases for DummyOperator and EmptyOperator
baeb1f9
@Lee-W
feat(AIR302): extend the following rules
016624d 
* ``airflow.operators.email_operator.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator``
* ``airflow.operators.email.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator``
@Lee-W
feat(AIR301): remove the following checks from AIR301 as they should …
25d9e8d 
…be recategorized as AIR302 due to rule change

* airflow.sensors.external_task_sensor.ExternalTaskMarker
* airflow.sensors.external_task_sensor.ExternalTaskSensor
* airflow.sensors.external_task_sensor.ExternalTaskSensorLink
* airflow.sensors.time_delta_sensor.TimeDeltaSensor
* airflow.operators.dagrun_operator.TriggerDagRunLink
* airflow.operators.dagrun_operator.TriggerDagRunOperator
* airflow.operators.python_operator.BranchPythonOperator
* airflow.operators.python_operator.PythonOperator
* airflow.operators.python_operator.PythonVirtualenvOperator
* airflow.operators.python_operator.ShortCircuitOperator
* airflow.operators.latest_only_operator.LatestOnlyOperator
* airflow.operators.bash_operator.BashOperator
* airflow.operators.branch_operator.BaseBranchOperator
@Lee-W
test(AIR301): clean up unused variable in test fixtures
5351a3d
@Lee-W
feat(AIR302): remove the following checks from AIR301 as they should …
1f320d5 
…be recategorized as AIR302 due to rule change

* airflow.sensors.date_time_sensor.DateTimeSensor
@Lee-W
test(AIR301): clean up unused test cases
7cd6cef
@Lee-W
feat(AIR302): extend the following rules
af589e7 
* airflow.sensors.external_task_sensor.ExternalTaskMarker → airflow.providers.standard.sensors.external_task.ExternalTaskMarker
* airflow.sensors.external_task_sensor.ExternalTaskSensor → airflow.providers.standard.sensors.external_task.ExternalTaskSensor
* airflow.sensors.external_task_sensor.ExternalTaskSensorLink → airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink
* airflow.sensors.time_delta_sensor.TimeDeltaSensor → airflow.providers.standard.sensors.time_delta.TimeDeltaSensor
* airflow.operators.dagrun_operator.TriggerDagRunLink → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
* airflow.operators.dagrun_operator.TriggerDagRunOperator → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator
* airflow.operators.python_operator.BranchPythonOperator → airflow.providers.standard.operators.python.BranchPythonOperator
* airflow.operators.python_operator.PythonOperator → airflow.providers.standard.operators.python.PythonOperator
* airflow.operators.python_operator.PythonVirtualenvOperator → airflow.providers.standard.operators.python.PythonVirtualenvOperator
* airflow.operators.python_operator.ShortCircuitOperator → airflow.providers.standard.operators.python.ShortCircuitOperator
* airflow.operators.latest_only_operator.LatestOnlyOperator → airflow.providers.standard.operators.latest_only.LatestOnlyOperator
* airflow.sensors.date_time_sensor.DateTimeSensor → airflow.providers.standard.sensors.DateTimeSensor
@Lee-W
test: reorg test cases
a252339
@Lee-W
fix(AIR302): fix S3KeySensor path error
8f37995
@Lee-W
style: add missed style
44e5cb6
@Lee-W Lee-W force-pushed the update-AIR302-303-rules branch from 37057cd to eaa3656 Compare April 7, 2025 01:46
@Lee-W Lee-W requested a review from ntBre April 7, 2025 01:47
@Lee-W
Copy link
Contributor Author

Lee-W commented Apr 7, 2025

I think this one is ready again. Thanks!

@MichaReiser MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Apr 7, 2025
@ntBre ntBre changed the title [airflow] update oudated AIR301, AIR302 rules [airflow] Update oudated AIR301, AIR302 rules Apr 7, 2025
ntBre
ntBre approved these changes Apr 7, 2025
View reviewed changes
Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ntBre ntBre merged commit 1e9e423 into astral-sh:main Apr 7, 2025
22 checks passed
dcreager added a commit that referenced this pull request Apr 7, 2025
@dcreager
Merge branch 'main' into dcreager/special-class
968e637 
* main: (42 commits)
  [playground] New default program (#17277)
  [red-knot] Add `--python-platform` CLI option (#17284)
  [red-knot] Allow ellipsis default params in stub functions (#17243)
  [red-knot] Fix stale syntax errors in playground (#17280)
  Update Rust crate clap to v4.5.35 (#17273)
  Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061)
  [ci] Fix pattern for code changes (#17275)
  [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123)
  [docs] fix formatting of "See Style Guide" link (#17272)
  [red-knot] Support stub packages (#17204)
  ruff_annotate_snippets: address unused code warnings
  [red-knot] Add a couple more tests for `*` imports (#17270)
  [red-knot] Add 'Format document' to playground (#17217)
  Update actions/setup-node action to v4.3.0 (#17259)
  Update actions/upload-artifact action to v4.6.2 (#17261)
  Update actions/download-artifact action to v4.2.1 (#17258)
  Update actions/setup-python action to v5.5.0 (#17260)
  Update actions/cache action to v4.2.3 (#17256)
  Update Swatinem/rust-cache action to v2.7.8 (#17255)
  Update actions/checkout action to v4.2.2 (#17257)
  ...
Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 8, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
eb27b06 
…#17123
@Lee-W Lee-W mentioned this pull request Apr 8, 2025
Merged
Lee-W added a commit to apache/airflow that referenced this pull request Apr 8, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
bcddef3 
…#17123 (#48634)
dcreager added a commit that referenced this pull request Apr 8, 2025
@dcreager
Merge branch 'main' into dcreager/legacy-typevar-instance
6284357 
* main: (222 commits)
  [playground] New default program (#17277)
  [red-knot] Add `--python-platform` CLI option (#17284)
  [red-knot] Allow ellipsis default params in stub functions (#17243)
  [red-knot] Fix stale syntax errors in playground (#17280)
  Update Rust crate clap to v4.5.35 (#17273)
  Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061)
  [ci] Fix pattern for code changes (#17275)
  [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123)
  [docs] fix formatting of "See Style Guide" link (#17272)
  [red-knot] Support stub packages (#17204)
  ruff_annotate_snippets: address unused code warnings
  [red-knot] Add a couple more tests for `*` imports (#17270)
  [red-knot] Add 'Format document' to playground (#17217)
  Update actions/setup-node action to v4.3.0 (#17259)
  Update actions/upload-artifact action to v4.6.2 (#17261)
  Update actions/download-artifact action to v4.2.1 (#17258)
  Update actions/setup-python action to v5.5.0 (#17260)
  Update actions/cache action to v4.2.3 (#17256)
  Update Swatinem/rust-cache action to v2.7.8 (#17255)
  Update actions/checkout action to v4.2.2 (#17257)
  ...
@BrewTestBot BrewTestBot mentioned this pull request Apr 10, 2025
Merged
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 29, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
78feffa 
…#17123 (#48634)

GitOrigin-RevId: bcddef3ca9aad6aa0efe2f2f9f8149733aa5784b
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 24, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
91dd81f 
…#17123 (#48634)

GitOrigin-RevId: bcddef3ca9aad6aa0efe2f2f9f8149733aa5784b
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 22, 2025
@Lee-W
docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…
034b18b 
…#17123 (#48634)

GitOrigin-RevId: bcddef3ca9aad6aa0efe2f2f9f8149733aa5784b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ntBre ntBre ntBre approved these changes

Assignees

@ntBre ntBre

Labels

preview Related to preview mode features rule Implementing or modifying a lint rule

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[airflow] Autofix is suggesting to replace with the same symbol

3 participants

@Lee-W @ntBre @MichaReiser